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TITLE OF THE INVENTION 
ENCODED -DATA CONVERTING APPARATUS AND METHOD FOR THE SAME 
5 BACKGROUND OF THE INVENTION 

Field of the Invention 

[0001] The present invention relates to an encoded-data 
converting apparatus and method for efficiently converting 
10 data encoded by first encoding into data encoded by second 

encoding. 

Description of the Related Art 

[0002] In recent years, cameras have become digital, and 
some of them have begun to employ moving-picture encoding 

15 methods. For example, digital still cameras employ JPEG 

(Joint Photographic Experts Group) encoding in order to 
record still pictures. Also, Motion JPEG which performs 
video recording by consecutively encoding still pictures is 
becoming widely used. 

20 [0003] In addition, with widespread use of personal 

computers, MPEG (Moving Picture Experts Group) -4 encoding, 
standardized by the International Organization for 
Standardization (ISO), has begun to be used as a method for 
exchanging and delivering video on the Internet . 

25 [0004] MPEG- 4 is an encoding method using inter-encoding. 
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and is optimal for use on the Internet, etc., since it is 
suitable particularly for low bit rates, compared with 
conventional MPEG-1 and MPEG- 2. 

[0005] Similarly, it is desirable to use data encoded by 
5 Motion JPEG, for example, on the Internet similarly to an 

MPEG- 4 bit stream. 

[0006] Since Motion JPEG only uses intra-encoding without 
using inter-encoding, simply using Motion JPEG results in a 
data amount that is too large to be delivered on the 

10 Internet or the like. 

[0007] Accordingly, a technique is required to convert 
Motion JPEG encoded data into an MPEG- 4 bit stream. First, 
the Motion JPEG encoded data is temporarily decoded back to 
pixel level. Then, motion compensation may need to be 

15 performed, after which the stream must be re-encoded, thus 

requiring extensive encoding conversion processing. Also, 
the re-encoding may cause additional image deterioration in 
the re-encoded image data. 

[0008] A technique of the related art for converting an 
20 MPEG-1 encoded bit stream into an MPEG- 4 bit stream is 

disclosed in, for example, Japanese Patent Laid-Open No. 
2001-238214. A technique of the related art for converting 
an MPEG- 2 bit stream into an MPEG- 4 bit stream is disclosed 
in, for example, Japanese Patent Laid-Open No. 2001-238218. 
25 However, a technique in which an encoding format only using 
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intra- encoding without using inter-encoding, such as Motion 
JPEG, is converted into an encoding format using inter- 
encoding, such as MPEG- 4, has not been proposed. 

5 SUMMARY OF THE INVENTION 

[0009] The present invention is made in view of the above 
circumstances, and it is an object of the present invention 
to provide an encoded-data converting apparatus for 
10 converting first encoded data into second encoded data at 

high speed and with a compact configuration without 
deteriorating image quality, and a method for the encoded- 
data converting apparatus. 

[0010] According to an aspect of the present invention, 
15 an encoded-data converting apparatus is provided which 

includes an encoded-data input unit for inputting first 
encoded data generated by using first encoding to encode 
image data, a header extracting unit for extracting header 
information included in the first encoded data, a header 
20 generating unit for generating header information based on 

second encoding from the header information included in the 
first encoded data, a conversion setting unit for setting-up 
variable-length-code conversion based on a predetermined 
parameter for use in the second encoding and an encoding 
25 parameter used in the first encoding, a frame-data 
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extracting unit for extracting encoded data in one frame 
from the first encoded data, a variable- length-code 
converting unit for converting a variable length code in the 
encoded data in the frame in accordance with the variable- 
5 length-code conversion set-up by the conversion setting unit, 

a DC-component converting unit for decoding DC components in 
the encoded data in the frame into quantized values and 
using the quantized values to perform predictive encoding 
based on the second encoding, and an encoded-data output 

10 unit for shaping the outputs of the header generating unit, 

the variable-length-code converting unit, and the DC- 
component converting unit into data, and outputting the data 
as data encoded by the second encoding. 
[0011] According to another aspect of the present 

15 invention, an encoded-data converting method is provided 

which includes an input step for inputting first encoded 
data generated by using first encoding to encode image data, 
a header extracting step for extracting header information 
included in the first encoded data, a header generating step 

20 for generating header information based on second encoding 

from the header information included in the first encoded 
data, a conversion setting step for setting-up variable- 
length-code conversion based on a predetermined parameter 
for use in the second encoding and an encoding parameter 

25 used in the first encoding, a frame-data extracting step for 
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extracting encoded data in one frame from the first encoded 
data, a variable-length-code converting step for converting 
a variable length code in the encoded data in the frame in 
accordance with the variable- length-code conversion set-up 
5 by the conversion setting step, a DC-component converting 

step for decoding DC components in the encoded data in the 
frame into quantized values and using the quantized values 
to perform predictive encoding based on the second encoding, 
and an encoded-data output step for shaping the outputs of 
10 the header generating step, the variable-length-code 

converting step, and the DC-component converting step into 
data, and output ting the data as data encoded by the second 
encoding. 

[0012] Further objects, features and advantages of the 
15 present invention will become apparent from the following 

description of the preferred embodiments with reference to 
the attached drawings . 



BRIEF DESCRIPTION OF THE DRAWINGS 

20 

[0013] Fig. 1 is a block diagram showing an encoded-data 
converting system according to a first embodiment of the 
present invention . 

[0014] Fig. 2 is a flowchart showing an encoded-data 
25 converting process according to the first embodiment of the 
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present invention . 

[0015] Fig. 3 is a flowchart showing an encoded-data 
converting process in units of frames according to the first 
embodiment of the present invention. 
5 [0016] Fig. 4 is a block diagram showing an encoded-data 

converting system according to a second embodiment of the 
present invention . 

[0017] Fig. 5 is a flowchart showing an encoded-data 
converting process according to the second embodiment of the 
10 present invention. 

[0018] Fig. 6 is a flowchart showing an encoded-data 
converting process in units of frames according to the 
second embodiment of the present invention. 

[0019] Fig. 7 is a block diagram showing an encoded-data 
15 converting system according to the third embodiment. 

[0020] Fig. 8 is an illustration of an example of the use 
of the memory according to the third embodiment of the 
present invention . 

[0021] Fig. 9 is a flowchart showing an encoded-data 
20 converting process in units of frames according to the third 

embodiment of the present invention. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 



25 



[0022] Embodiments of the present invention are described 



below with reference to the accompanying drawings . 
First Embodiment 

[0023] Fig. 1 is a block diagram showing an encoded-data 
converting system according to a first embodiment of the 
present invention . 

[0024] The encoded-data converting system in Fig. 1 
consists of an encoded-data converting apparatus 1, and 
storage units 2 and 3. The storage unit 2 stores Motion - 
JPEG encoded data, for example, and the storage unit 3 
stores MPEG- 4 encoded data, for example. Although, in the 
first embodiment, the above encoding methods are described 
as examples, the present invention is not limited to the 
above encoding methods, but may be applied to an encoded- 
data converting system and method in which data encoded by 
intra-encoding is converted into data encoded by inter- 
encoding . 

[0025] Referring to Fig. 1, a separator 4 interprets and 
separates Motion- JPEG encoded data into a header and encoded 
data in units of frames, and a header analyzer 5 interprets 
and separates the header into header image information 
describing an image itself, and parameters required for 
encoding. An image information encoder 6 encodes the header 
image information describing the image and creates a header 
based on another encoding scheme such as MPEG- 4. An 
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encoding parameter encoder 7 encodes the aforementioned 
parameters, and creates a header based on MPEG- 4. 
[0026] An MPEG- 4 variable length code (VLC) memory 8 
stores a variable -length- code encoding table for use in 
5 intra-macroblock encoding based on MPEG- 4. A code 

conversion setting unit 9 generates a code conversion table 
using the parameters and the variable- length-code encoding 
table . 

[0027] A memory 10 temporarily stores JPEG encoded data 

10 received from separator 4 into frames and sends the encoded 

data to a post stage in units of blocks. A VLC decoder 11 
decodes variable length codes of DC components in each block. 
A DC prediction unit 12 performs prediction from surrounding 
DC components in accordance with a prediction method based 

15 on MPEG-4. Based on MPEG-4, a DC encoder 13 performs 

variable length encoding on prediction errors on the DC 
components obtained by the DC prediction unit. 
[0028] Based on information in the conversion table 
generated by the code conversion setting unit 9, a code 

20 converter 14 converts the JPEG encoded data of AC components 

into MPEG-4 encoded data. A shaping unit 15 shapes the 
outputs of the image information encoder 6, the encoding 
parameter encoder 7, the DC encoder 13, and the code 
converter 14 into encoded data consistent with the MPEG-4 

25 format. 
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[0029] A code converting process of the encoded-data 
converting system having the above configuration is 
described below. 

[0030] Before performing the conversion, the components 
5 of the encoded-data converting system are initialized. 

Specifically, the memory 10 is cleared and parameters used 
in the components are reset. Also, in the storage unit 2, 
Motion-JPEG encoded data to be converted is selected and a 
start position at which the selected data is to be read is 
10 set. In the storage unit 3, a start position of writing 

into a writable area in unused areas is set. 

[0031] When the initialization ends, Motion-JPEG encoded 
data is input from the storage unit 2 to the encoded-data 
converting apparatus 1. In the encoded-data converting 
15 apparatus 1, the separator 4 reads and separates the first 

JPEG encoded data item into encoded header data and encoded 
frame data. The encoded header data is stored in the header 
analyzer 5, and the encoded data in the frame is stored in 
the memory 10. 

20 [0032] The header analyzer 5 interprets JPEG encoding 

headers. In JPEG encoding, initially, encoded data of an 
image layer is input. A detailed description of JPEG 
encoding is omitted since its details are described in the 
ISO 10918-1 standard. Various types of table data, are 

25 described, that follow a marker code representing the start 
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of an image . 

[0033] The table data is described because JPEG encoding 
does not have any default values concerning a quantization 
matrix and the Huffman table. The table data contains 
5 encoding parameters for encoding and decoding. The encoding 

parameters are input into the encoding parameter encoder 7 . 
[0034] In this case, an exemplary quantization matrix has 
64 quantization table elements subsequent to a define 
quantization table (DQT) marker code defining the 

10 quantization table. Accordingly, the quantization table 

elements are read, or the quantization table elements are 
read in accordance with numbers of the quantization table. 
The read quantization table elements are input to the 
encoding parameter encoder 7. Also, regarding the Huffman 

15 table, Huffman codes follow a Define Huffman Table (DHT) 

marker code defining the Huffman table. The Huffman codes 
are read and input to the code conversion setting unit 9. 
[0035] The encoding parameter encoder 7 compares the 
input quantization table elements and a default quantization 

20 matrix on an MPEG-4 intra-block. 

[0036] A detailed description of the MPEG-4 standard is 
omitted here since its details are described in the ISO 
14496-2 standard. 

[0037] When the input quantization table elements differ 
25 from the default quantization matrix, the 
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load_intra_quant_mat code of a video object layer (VOL) 
header is set to be 1, and a quantization table element of 
an AC component is described as a load_intra_mat code. Also, 
the vop_coding_type code of a video object plane (VOP) 
5 header is always set to be "00" representing intra-coded( I ) . 

An ac_pred_flag code describing whether AC prediction 
encoding in units of macroblocks is performed is set to be 0. 
[0038] The code conversion setting unit 9 performs 
Huffman table processing. The following description takes, 
10 for instance, a case in which the Huffman table in JPEG is 

as described in table K.5 of Annex K of the ISO 10918-1 
standard. 

[0039] At first, codes whose run lengths are zeroes are 
partly extracted in the following Table 1 . 

15 



Table 1 



RUN LENGTH 


SIZE 


CODE LENGTH 


CODE 


0 


1 


2 


00 


0 


2 


2 


01 


0 


3 


3 


100 


0 


4 


4 


1011 


0 


5 


5 


11010 


0 


6 


7 


1111000 


0 


7 


8 


11111000 


0 


8 


10 


1111110110 


0 


9 


16 


1111111110000010 


0 


10 


16 


1111111110000011 



[0040] Also, the Huffman table in MPEG- 4 encoding is 
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uniquely set as described in Table B-16 of Annex B of the 
ISO 14496-2 standard, and its values are stored in the MPEG- 
4 VLC memory 8. The MPEG-4 encoding is a three-dimensional 
VLC method in which a variable length code is assigned to a 
5 set of (LAST, RUN, LEVEL), in contrast to two-dimensional 

Huffman coding. 

[0041] For describing Huffman codes whose LAST are zeroes, 

codes whose run lengths are zeroes and whose LASTs are 
zeroes are partly shown in the following Table 2. 

10 



Table 2 



LAST 


RUN 


LEVEL 


CODE LENGTH 


CODE 


0 


0 


1 


2 


10 


0 


0 


2 


3 


110 


0 


0 


3 


4 


1111 


0 


0 


4 


5 


01101 


0 


0 


5 


5 


01100 


0 


0 


6 


6 


010101 


0 


0 


7 


6 


010011 


0 


0 


8 


6 


010010 


0 


0 


9 


7 


0010111 


0 


0 


10 


8 


00011111 



[0042] Also, since, when an EOB follows the Huffman codes 
input in JPEG encoding, LASTs are Is, codes whose run 
15 lengths are 0s and whose LASTs are Is are partly shown in 

the following Table 3. 
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Table 3 



T ACT 


"PTTM 


T T?\7"FT 


UUL/Ei i_i Hi IN VJ x n 




1 
J- 


n 


1 

j. 


4 


0111 ; 


1 


0 


2 


6 


001100 


1 


0 


3 


8 


00010110 


1 


0 


4 


9 


000010111 


1 


0 


5 


10 


0000000110 


1 


0 


6 


11 


00000000101 


1 


0 


7 


11 


00000000100 


1 


0 


8 


12 


000001011001 


1 


0 


9 


12 


000001100111 


1 


0 


10 


14 


00000110001100 



[0043] Here, in the case of (1, 0, 9) and (1, 0, 10), 
escape mode 1 is applied. 
5 [0044] The MPEG- 4 encoding has three escape modes. 

[0045] In escape mode 1, after LMAX is subtracted from 
the absolute value of LEVEL of (LAST, RUN , LEVEL), three- 
dimensional VLC is used to perform variable length encoding 
(Esc + "0" + three-dimensional VLC). The value of LMAX is 
10 the maximum value of LEVEL corresponding to (LAST, RUN) . 

This is shown in Table B-19 of Annex B of the ISO 14496-2 
standard. 

[0046] When the variable length encoding cannot be 
performed in escape mode 1, in escape mode 2, after the 
15 value (RMAX + 1) is subtracted from RUN of (LAST, RUN, 

LEVEL), three-dimensional VLC is used to perform variable 
length encoding (Esc + "10" + three-dimensional VLC). The 



- 14 - 



value of RMAX is the maximum value of RUN corresponding to 
(LAST, RUN). This is shown in Table B-21 of Annex B of the 
ISO 14496-2. 

[0047] When variable length encoding cannot be performed 
5 even in escape mode 2, in escape mode 3, fixed length 

encoding (LAST: 1 bit; RUN: 6 bits; LEVEL: 12 bits) is 
performed (Esc + "11" + fixed length encoding). 



Table 4 







JPEG 


MPEG- 4 


RUN 
LENGTH 


SIZE 


CODE 
LENGTH 


CODE 


CODE 
LENGTH 


CODE 


0 


1 


2 


00 


2 


10 


0 


2 


2 


01 


3 


110 


0 


3 


3 


100 


4 


1111 


0 


4 


4 


1011 


5 


01101 


0 


5 


5 


11010 


5 


01100 


0 


6 


7 


1111000 


6 


010101 


0 


7 


8 


11111000 


6 


010011 


0 


8 


10 


1111110110 


6 


010010 


0 


9 


16 


11111111100000 
10 


7 


0010111 


0 


10 


16 


11111111100000 

11 


8 


00011111 



[0048] Also, when an EOB follows, a table for converting 
Huffman codes in JPEG encoding into Huffman codes in MPEG- 4 
encoding is created as shown in the following Table 5, based 
on the above Tables 1 and 3 . 



15 
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Table 5 







JPEG 


MPEG- 4 


"PTTM 

LENGTH 


SIZE 


CODE 


CODE 


CODE 

T T7M/~ ,r PU 

LiCiJNlal ii 


CODE 


o 


1 


2 


00 


4 


0111 


0 


2 


2 


01 


6 


001100 


0 


3 


3 


100 


8 


00010110 


0 


4 


4 


1011 


9 


000010111 


0 


5 


5 


11010 


10 


0000000110 


0 


6 


7 


1111000 


11 


00000000101 


0 


7 


8 


11111000 


11 


00000000100 


0 


8 


10 


1111110110 


12 


000001011001 


0 


9 


16 


11111111100000 
10 


12 


000001100111 


0 


10 


16 


11111111100000 

11 


14 


00000110001100 



[0049] By comparing the Huffman table in MPEG- 4 encoding 
which is stored in the MPEG-4 VLC memory 8, the code 
5 conversion setting unit 9 similarly creates a conversion 

table for other Huffman codes. For brevity of description, 
a conversion table created when no EOB follows is 
hereinafter referred to as "conversion table A", and a 
conversion table created when an EOB follows is hereinafter 
10 referred to as "conversion table B" . These conversion 

tables are input to the code converter 14. 

[0050] Subsequently, the header analyzer 5 interprets the 
header of the first frame of the JPEG encoded data. The 
frame header includes image-related information such as a 
15 sampling precision of the input image, the number of lines, 

and the number of samples per line. These pieces of the 
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information are input to the image information encoder 6 . 
[0051] Based on the above pieces of the information, the 
sampling precision of the input image, the number of lines, 
and the number of samples per line, the image information 
5 encoder 6 determines codes for use in MPEG- 4 encoding. 

Regarding the sampling precision, in the case of 8 bits, the 
not_8_bit code of a VOL header is set to be 0 , and in the 
case of not 8 bits, the not_8_bit is set to be 1 . Then, by 
setting a quant_precision code and a bits per pixel code, as 

10 required, codes are generated. 

[0052] Since the number of lines and the number of 
samples per line represent the vertical and horizontal sizes 
of the image, respectively, the values of both are set in 
the video_object_layer_height code and 

15 video_ob ject_layer_width code of the VOL header, whereby 

codes are generated. Obviously, the shape of the image is 
rectangular. Thus, a video_ob ject__layer_shape code is set 
to "rectangular". By setting other codes in MPEG-4 encoding, 
a visual object sequence (VOS) header and a visual object 

20 (VO) header, and a vol header, as required, codes are 

generated. The generated codes are input to the shaping 
unit 15. The shaping unit 15 shapes the input codes into 
header data, that is, VOS (visual object sequence), VO 
(visual object), and VOL (video object layer), and stores 

2 5 the shaped data in a predetermined area of the storage unit 
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3. 

[0053] The encoded data in the frame, separated in the 
separator 4, is stored in the memory 10. Its DC components 
are input in units of blocks to the VLC decoder 11, and its 
5 AC components are input in units of blocks to the code 

converter 14. 

[0054] DC component processing is now being described 
below. 

[0055] The VLC decoder 11 decodes DC component codes and 
10 inputs prediction error values of quantized DC components to 

the DC prediction unit 12. Using DC components of the 
previous block and the input prediction error values, the DC 
prediction unit 12 restores and stores quantized values of 
DC components, and DC components in MPEG-4 encoding are 
15 predicted based on DC components of surrounding blocks. 

Based on a DC component in a block in the same position on 
an adjacent line and a DC component of an adjacent block, a 
predicted DC error is found and input to the DC encoder 13. 
[0056] The DC encoder 13 generates a code by encoding the 
20 input predicted DC error in accordance with a DC component 

encoding process based on MPEG-4. The generated code is 
input to the shaping unit 15. 

[0057] Also, the code converter 14, into which the AC 
component code is input, performs conversion in accordance 
25 with the conversion table generated by the code conversion 
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setting unit 9. When no EOB follows the read Huffman codes, 
the code converter 14 performs conversion into Huffman codes 
in MPEG- 4 encoding, in accordance with conversion table A, 
for example. When an EOB follows, the code converter 14 
performs conversion in accordance with conversion table B, 
for example, and proceeds to processing of the next block. 
The generated code is input to the shaping unit 15. 
[0058] The codes, generated by the image information 
encoder 6, the encoding parameter encoder 7, the DC encoder 
13, and the code converter 14, are rearranged in MPEG- 4 
encoding format by the shaping unit 15. The rearranged 
codes are output and sequentially written in predetermined 
areas of the storage unit 3. 

[0059] The above -described process is performed for all 
the frames of the JPEG encoded data stored in the storage 
unit 2. Frame-data reading and code conversion are 
performed, and the conversion results are output before 
terminating the encoded-data converting process. 
[0060] The above -described encoded-data converting 
process is described below with reference to the flowcharts 
shown in Figs . 2 and 3 . 

[0061] Fig. 2 is a flowchart showing an encoded-data 
converting process performed in the first embodiment of the 
present invention . 

[0062] Referring to Fig. 2, in step SI, encoded data of 
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an image layer such as JPEG encoded data is read. 
[0063] In step S2 , from the encoded data, codes on a 
quantization table are extracted and compared to a 
quantization matrix based on MPEG- 4 encoding, and codes are 
5 generated, if required. 

[0064] In step S3, by extracting and comparing Huffman- 
table-related codes with a Huffman table based on MPEG-4 
encoding, a conversion table is created. 
[0065] In step S4 , frame data is read from the JPEG 
10 encoded data and is analyzed. 

[0066] In step S5, from the result of the analysis, etc., 
image information is obtained, and VOS, VO, and VOL header 
data items are created and output. 

[0067] In step S6, encoded data concerning pixels in the 
15 frame is converted. The conversion process is described 

later with reference to the flowchart shown in Fig. 3. 
[0068] In step S7, it is determined whether all the frame 
data items have been processed. When there is an 
unprocessed frame data item, the process proceeds to step S8 . 
20 [0069] In step S8 , JPEG-encoded frame data is read. 

Returning to step S6, the frame data is converted. In step 
S7, if the process has determined that all the frame data 
items have been processed, the process is terminated. 
[0070] Fig. 3 is a flowchart showing the encoded-data 
25 converting process in units of frames, which is shown in 
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step S6 of Fig. 2. 

[0071] In Fig. 3, in step S21, a VOP header is created 
from the image information, etc. , and is output. 
[0072] In step S22, termination determination is 
5 performed by determining whether or not all the macroblocks 

of the frame have been processed. If the macroblocks have 
been processed, the process is terminated. If not, the 
process proceeds to step S23. 

[0073] In step S23, it is determined whether or not all 

10 the blocks of the macroblocks have been processed. If the 

blocks have been processed, the process returns to step S22. 
If not, the process proceeds to step S24. 
[0074] In step S24, for a DC component in the block, 
decoding on a prediction error is performed. In addition, 

15 reproduction of a quantized value of the DC component based 

on prediction in JPEG, and prediction and encoding in MPEG- 4 
encoding are performed. The obtained result is output. 
[0075] In step S25, by detecting an EOB code, it is 
determined whether or not all AC components have been 

20 converted. This determines termination of processing in 

units of blocks. If all the AC components have been 
converted, the process returns to step S23. If not, the 
process proceeds to step S26. In step S26, a Huffman code 
representing a run length of zero is extracted. 

25 [0076] In step S27, it is determined whether or not a 
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code next to the extracted code is an EOB. When the code 
next to the extracted code is not an EOB, the process 
proceeds to step S28. In step S28, the code is converted by 
using Table A and is output. In step S29, the code is 
5 converted by using Table B. 

[0077] As described above, according to the first 
embodiment, data encoded by a first encoding scheme such as, 
JPEG encoding, can be converted into data (I-VOP) encoded by 
a second encoding scheme such as MPEG- 4 encoding. Among 

10 other advantages, such conversion can be obtained without 

reproducing pixel levels and without deterioration. Further 
yet, compared with JPEG encoding, adaptive prediction of the 
DC component is performed in MPEG- 4 encoding. Thus, it is 
expected that the encoding efficiency increases. Also, very 

15 high speed processing can be performed since tables are used 

to perform code conversion. Moreover, the need for a memory 
for storing pixel levels is eliminated. Accordingly, 
complexity of processing can be reduced, thus enabling a 
reduction in apparatus cost . 

20 

Second Embodiment 

[0078] A second embodiment of the present invention is 
described below. 

[0079] Fig. 4 is a block diagram showing an encoded-data 
25 converting system according to the second embodiment of the 
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present invention. Also, in the second embodiment, a case 
in which Motion- JPEG encoded data is an input and MPEG- 4 
encoded data is an output is described. However, that the 
present invention is not limited to the above case is 
5 similar to the first embodiment. In Fig. 4, by denoting 

components identical to those in Fig. 1 showing the first 
embodiment by identical reference numerals, a detailed 
description of each component is omitted. 

[0080] Fig. 4 shows an encoded-data converting apparatus 
10 101 according to the second embodiment. The encoded-data 

converting apparatus 101 includes a VLC decoder 102 for 
decoding a variable length code of an AC component in each 
block, and a motion compensator 103 for performing motion 
compensation in units of macroblocks by using quantized 
15 values. 

[0081] An encoder 104 encodes a prediction error of a 
macroblock, and motion vectors which are obtained by the 
motion compensator 103, into a variable length code in 
accordance with MPEG- 4 encoding. A memory 105 stores 
20 quantized values for each block. A shaping unit 106 shapes 

the outputs of an image information encoder 6, an encoding 
parameter encoder 7, a DC encoder 13, a code converter 14, 
and an encoder 104 into encoded data complying with the 
MPEG- 4 format. 

25 [0082] Code conversion of the encoded-data converting 
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system having the above-described configuration is described 
below. 

[0083] Before performing the conversion, the components 
of the encoded-data converting system are initialized. 
Specifically, the memory 10 and the memory 105 are cleared 
and parameters used in the components are reset. Also, in 
the storage unit 2, Motion- JPEG encoded data to be converted 
is selected and a start position at which the selected data 
is read is set. In the storage unit 3, a start position of 
writing into a writable area in unused areas is set. 
[0084] When the initialization ends. Motion- JPEG encoded 
data is input from the storage unit 2 to the encoded-data 
converting apparatus 101. In the encoded-data converting 
apparatus 101, the separator 4 reads and separates the first 
JPEG encoded data item into a header and encoded data in 
units of frames. The header is stored in the header 
analyzer 5, and the encoded data in the frames is stored in 
the memory 10. 

[0085] Similarly to the first embodiment, the header 
analyzer 5 interprets the JPEG headers and extracts encoding 
parameters and image information. An encoding parameter 
encoder 7 and the image information encoder 6 create and 
output header data to the shaping unit 106. A code 
conversion setting unit 9 creates and inputs a conversion 
table to a code converter 14. 



[0086] Next, frame-unit processing is described below. 
In the second embodiment, I-VOP representing an in-frame 
encoding mode in the MPEG- 4 encoding, and P-VOP representing 
an interframe encoding modes are generated. Although the 
following description is directed to a case in which I-VOP 
is included at regular intervals, the case has no limitation 
caused by intervals and frame-mode determination. 
[0087] At first, a case in which the encoding mode in the 
frame is I-VOP is described below. In the encoding 
parameter encoder 7 , the vop_coding_type code of a VOP 
header is set to be "00" representing intra-coded(I) . 
Similarly to the first embodiment, the frame data in the 
frame, separated by the separator 4, is stored in the memory 
10. DC components are input in units of blocks to the VLC 
decoder 11 and the VLC decoder 102, and AC components are 
input to the code converter 14 and the VLC decoder 102. 
[0088] Conversion of the codes of the DC components is 
performed by the VLC decoder 11, the DC prediction unit 12, 
and the DC encoder 13, similarly to the first embodiment. 
[0089] Also, conversion of the AC components is performed 
by the code converter 14, similarly to the first embodiment. 
The converted encoded data is output through the shaping 
unit 106. 

[0090] Regarding encoded data input from the memory 10 to 
the VLC decoder 102 in units of blocks, the DC components 
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and the AC components are decoded into variable length data, 
and quantized values are reproduced and input to the motion 
compensator 103. The motion compensator 103 does not 
operate based on I-VOP, and the quantized values are 
5 directly stored in the memory 105. The encoder 104 also 

does not operate based on I-VOP. 

[0091] Next, a case in which the encoding mode in the 
frame is P-VOP is described below. 

[0092] The encoding parameter encoder 7 sets the 
10 vop_coding_type of the VOP header to be "01" representing 

predictive-coded(P) . The encoded data in the frame, stored 
in the memory 10, is read in units of blocks and is input to 
the VLC decoder 102. For each of DC components and AC 
components, variable length data is decoded and quantized 
15 values are reproduced and input to the motion compensator 

103. The motion compensator 103 compares the input 
quantized values and the quantized values stored in units of 
blocks in the memory 105. The comparison may be performed 
only for, for example, luminance. In addition, chromaticity 
20 may be added in the comparison. 

[0093] Motion compensating processing of the motion 
compensator 103 is described below. 

[0094] Although the following description is directed to 
a case in which an image has a format of CIF (352 by 288), 
25 definitely, the image format is not limited to this case. 
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[0095] It is assumed that an input block has coordinates 
(I, J) in an image and a quantized value is represented by 
G(x, y) # where I represent a value of 0 to 21, and J 
represents a value of 0 to 17. For a quantized value 
5 Pi,j(x,y)(i = 0 to 21, j = 0 to 17), the sum of the absolute 

values of differences is calculated in accordance with the 
following expression: 



Sij = Z x=0 ,72y=o,7|G(x,y)-Pi, j(x,y) | (i) 



10 



[0096] Also, with four input blocks treated as a 
macroblock unit, each Sij is collectively treated, the sums 
of the absolute values of differences are found for 
macroblocks, and the least sum of the sums is selected. 

15 This causes the state of the image to be equivalent to a 

state in which motion compensation in units of eight pixels 
is performed. A relative position with respect to the 
selected block is used as a motion vector, and a difference 
therefrom is output as a prediction error to the encoder 104, 

20 As mentioned above, in the second embodiment, the motion 

vector is treated in units of 16 pixels. In addition, by 
selecting the inter- 4MV mode which sets a motion vector for 
each block, motion compensation for each block may be 
performed. In this case, 8-pixel-precision motion 

25 compensation can be performed. 
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[0097] The encoder 104 encodes the motion vector and the 
prediction error between the quantized values . The obtained 
encoded data is input to the shaping unit 106. 
[0098] When the encoding mode in the frame is I-VOP, the 
5 shaping unit 106 rearranges the codes generated by the image 

information encoder 6, the encoding parameter encoder 7, the 
DC encoder 13, and the code converter 14 in accordance with 
the MPEG -4 format. 

[0099] Also, when the encoding mode in the frame is P-VOP, 

10 the shaping unit 106 rearranges the codes generated by the 

image information encoder 6, the encoding parameter encoder 
7, and the encoder 104 in accordance with the MPEG- 4 format. 
The rearranged codes are output to the storage unit 3 and 
are sequentially written in predetermined areas. 

15 [0100] Frame data of all the frames of the JPEG encoded 

data stored in the storage unit 2 is read and converted. 
After the data is output before the process is terminated. 
[0101] The flow of the above encoded-data converting 
process is described below with reference to the flowcharts 

20 shown in Figs. 5 and 6. 

[0102] Fig. 5 is a flowchart showing an encoded-data 
converting process in the second embodiment. By denoting 
steps identical to those in the above first embodiment by 
identical reference numerals, a detailed description of each 

25 step is omitted. 
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[0103] When it is determined in step S7 that there is 
unprocessed frame data, the process proceeds to step S100. 
In step S100, the process determines one of I-VOP and P-VOP 
as the encoding mode. For example, the encoding mode may be 
5 determined so as to be I-VOP at regular intervals. After 

that, in step S8, frame data is read similarly to the first 
embodiment. Returning to step S6, the code of the frame 
data is converted in accordance with the encoding mode . 
[0104] Fig. 6 is a flowchart showing a process for the 
10 encoded data conversion in units of frames in step S6 in Fig. 

5. By denoting steps identical to those in the first 
embodiment by identical reference numerals, a detailed 
description of each step is omitted. 

[0105] In step S121, from the image information, etc., a 
15 VOP header is created and output. At this time, a 

vop_coding_type code is set to be "00" when the I-VOP 

encoding is selected in step S100 in Fig. 5, and is set to 

be "01" when the P-VOP encoding is selected. 

[0106] In step S122, a processing type is selected 
20 depending on the encoding mode determined in step S100 in 

Fig. 5. When the encoding mode is I-VOP, the process 

proceeds to step S22, and when the encoding mode is P-VOP, 

the process proceeds to step S124. 

[0107] When the encoding mode is I-VOP, steps S24 to S26 
25 are performed before the process proceeds to step S123. In 
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step S123, a quantized value is obtained by decoding DC- 
component and AC -component variable length data and is 
stored in the memory 105. After that, similarly to the 
first embodiment, AC-component conversion is performed in 
5 steps S27 to S29. 

[0108] When the encoding mode is P-VOP, in step S124, 
termination determination is performed similarly to step S22. 
When all the macroblocks have not been processed yet, in 
step S125, a quantized value is obtained by decoding DC- 
10 component and AC-component variable length data in one 

macroblock, and is stored in the memory 105. The process 
proceeds to step S126. 

[0109] In step S126, motion vector retrieval and 
prediction error calculation are performed by referring to 
15 the quantized value. In step S127, the obtained motion 

vector and prediction error are encoded by MPEG- 4 encoding 
and are output. The process returns to step S124 in order 
to process the next macroblock. 

[0110] As described above, according to the second 
20 embodiment, data encoded by JPEG encoding, which is an in- 

frame encoding method, can be converted into data (I-VOP and 
P-VOP) encoded by MPEG-4 encoding, which is an inter-frame 
encoding method, without reproducing pixel levels and 
deterioration. As a result, compared with JPEG encoding, it 
25 is expected that, in MPEG-4 encoding, motion compensation 
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greatly increases the encoding efficiency. Also, very high 
speed processing can be performed since code conversion can 
also be performed by referring to tables. It is not 
necessary to use a memory for storing pixel levels. Thus, 
5 the configuration required for processing can be reduced, 

thus enabling a reduction in apparatus cost. Moreover, the 
motion vector retrieval requires a very small number of 
arithmetic operations, thus enabling a reduction in 
apparatus cost. In addition, the motion vector retrieval 
10 requires a very small number of arithmetic operations, thus 

enabling high speed retrieval. 

Third Embodiment 

[0111] A third embodiment of the present invention is 
15 described below. The third embodiment shows an example of 

the encoded-data converting process which is realized by 
software . 

[0112] Fig. 7 is a block diagram showing an encoded-data 
converting apparatus in the third embodiment. 

20 [0113] Referring to Fig. 7, a central processing unit 

(CPU) 300 controls the entirety of the encoded-data 
converting apparatus, and a memory 301 provides the storage 
area required for arithmetic operations by an operating 
system (OS) necessary for the apparatus control and by 

2 5 software. 
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[0114] The apparatus includes a bus 302 for connecting 
various types of devices and exchanging data and control 
signals, an operation unit 303 for starting the apparatus, 
setting various conditions, and commanding playback, and a 
5 storage unit 304 for storing software. 

[0115] Also, a storage unit 305 stores image data, and a 
camera 306 captures images. Storage units 304 and 305 may 
also be formed by media which is removable from the system. 
The apparatus also includes a monitor 307 for displaying an 
10 image, a communication link 308 including a local area 

network, a public circuit, a wireless link, airwaves, etc., 
a communication interface (I/F) 309 for 

transmitting/receiving image data through the communication 
link 308. 

15 [0116] In the memory 301, the OS for operating various 

types of software, and software to be operated are stored. 
The memory 301 includes a JPEG encoded data area for storing 
input JPEG encoded data, an MPEG- 4 encoded data area for 
storing MPEG- 4 encoded data to be output, and a working area 

20 for storing parameters, etc., such as quantized values 

obtained by decoding. 

[0117] An encoded data transmitting process in which, in 
the above configuration. Motion- JPEG encoded data stored in 
the storage unit 305 is converted into MPEG- 4 encoded data 
25 and is transmitted is described below. 
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[0118] Before the encoded data transmitting process is 
performed, the operation unit 303 instructs the entirety of 
the apparatus to start, and each component of the apparatus 
is initialized. Then, the software stored in the storage 
5 unit 304 is loaded into the memory 301 through the bus 302, 

whereby the software is started. 

[0119] Fig. 8 shows a state in which the memory 301 is 
used. As shown in Fig. 8, in the memory 301, the OS for 
controlling the entirety of the apparatus and operating the 
10 types of software, communication software for controlling 

communication, and encoded-data converting software are 
loaded. 

[0120] After the apparatus is started, the communication 
software in the memory 301 is started to establish a 
15 communication path with another apparatus through the 

communication link 308. 

[0121] Encoded data conversion by the CPU 300 is 
described below. The second embodiment shown in Fig. 5 
applies to image -information extraction, encoding-parameter 
20 extraction, image- information encoding, encoding-parameter 

encoding, and creation of a header for MPEG- 4 encoded data. 
Accordingly, the third embodiment is described with 
reference to Fig. 5. 

[0122] In step SI, encoded data of an image layer of JPEG 
25 encoded data is read and stored in the JPEG encoded data 
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area of the memory 301. 

[0123] In step S2, from the encoded data of the image 
layer stored in the JPEG encoded data area of the memory 301, 
a code, which is an encoding parameter relating to a 
5 quantization table, is extracted and compared with an MPEG- 4 

encoding quantization matrix, A code is generated, as 
required, and is stored in the working area. 
[0124] In step S3, from the encoded data in the JPEG 
encoded data area of the memory 301, a code relating to the 
10 Huffman table is extracted and is compared with an MPEG- 4 

Huffman table. A conversion table is created and stored in 
the working area. 

[0125] In step S4, from the JPEG encoded data in the JPEG 
encoded data area of the memory 301, frame data is read and 
15 its header is analyzed. The obtained image information is 

stored in the working area of the memory 301. 

[0126] In step S5, VOS, VO, and VOL header data items are 
created from the image information and encoding parameters, 
etc. , and are stored in the MPEG- 4 encoded data area of the 
20 memory 301. 

[0127] In step S6, encoded data of pixels in the frame is 
converted. 

[0128] In step S7 , it is determined whether all the frame 
data items have been processed. When there is an 
25 unprocessed frame data item in the JPEG encoded data area of 
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the memory 301, the process proceeds to step S100. 

[0129] In step S100, it is determined which of I-VOP and 

P-VOP is the encoding mode. 

[0130] In step S8, JPEG encoded frame data is read from 
5 the JPEG encoded data area of the memory 301* 

[0131] In step S6 , encoded data of pixels in the frame is 
performed and the converted data is sequentially stored in 
the MPEG- 4 encoded data area. After that, the process 
proceeds to step S7. This conversion is described later 

10 with reference to Fig. 9. 

[0132] When it is determined in step S7 that all the 
frame data items have been processed, the process is 
terminated. The encoded data in the MPEG- 4 encoded data 
area of the memory 301 is read if needed, and is output by 

15 the communication software to the communication link 308 

through the communication interface 309. The communication 
software deletes the output data in the MPEG- 4 encoded data 
area of the memory 301. 

[0133] Next, the encoded data conversion in units of 
20 frames in step S100 in the third embodiment is described 

below with reference to the flowchart shown in Fig. 9. 
[0134] Referring to Fig. 9, in step S201, a VOP header is 
created from the image information and encoding parameters 
stored in the working area of the memory 301, and the 
25 encoding mode determined in step S6 in Fig. 5 is described 
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in a vop_coding_type code. The created VOP header is stored 
in the MPEG-4 encoded data area of the memory 301. The 
process proceeds to step S202. 

[0135] In step S202, it is determined whether or not all 
5 the macroblocks of the frame have been processed. If they 

have been processed, it is determined that the process of 
the frame ends. If it is not determined, the process 
proceeds to step S203. 

[0136] In step S203, a process type is selected depending 

10 on the encoding mode determined in step S6 . The process 

proceeds to step S204 when the encoding mode is I -VOP, and 
proceeds to step S212 when the encoding mode is P-VOP. 
[0137] When the encoding mode is I -VOP, in step S204, the 
header of a macroblock as an intra-macroblock is created and 

15 stored in the MPEG-4 encoded data area. The process 

proceeds to step S205. In step S205, it is determined 
whether or not all the blocks of the macroblock have already 
been processed. If it is affirmatively determined, the 
process returns to step S202. If it is negatively 

20 determined, the process proceeds to step S206. 

[0138] In step S206, decoding of a prediction error on a 
DC component of the block, reproduction of a quantized value 
of the DC component based on prediction in JPEG encoding, 
and prediction and encoding in MPEG-4 encoding are performed. 

25 The encoded data is stored in the MPEG-4 encoded data area. 
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and the DC component is stored in a corresponding block 
position in a portion of the working area for storing a 
quantized value. The process proceeds to step S207. 
[0139] In step S207, by detecting an EOB code, it is 
5 determined whether or not all AC components have been 

converted. This determines termination of the process in 
units of blocks. If the AC components have been processed, 
the process proceeds to step S210. If it is negatively 
determined, the process proceeds to S208. 

10 [0140] In step S208, a Huffman code representing a run 

length of zero is extracted before the process proceeds to 
step S209. In step S209, the quantized value is reproduced 
by decoding the input code, and is stored in a corresponding 
block position of the portion of the working area for 

15 storing quantized values. The process returns to step S207. 

[0141] When it is determined in step S207 that 
reproduction of block quantized values ends, in step S210, a 
DC component and an AC component are predicted in 
coefficient prediction in MPEG- 4 encoding from quantized 

20 values of surrounding blocks which are stored in the working 

area of the memory 301. The process proceeds to step S211. 
[0142] In step S211, the prediction error and the 
remaining AC component are encoded and stored in the MPEG- 4 
encoded data area. The process proceeds to step S205. In 

25 step S205, when all the blocks of the macroblock have been 
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processed, the process returns to step S202 in order to 
process the next macroblock . 

[0143] Also, when the encoding mode is P-VOP in step S203, 

in step S212, a quantized value is reproduced by decoding 
5 the input macroblock code and is stored in a corresponding 

block position in the portion of the working area of the 
memory 301 for storing quantized values. The process 
proceeds to step S213. 

[0144] In step S213, motion- vector retrieval in units of 

10 8 pixels is performed by referring to the quantized value in 

the previous frame which is stored in the working area, and 
the obtained motion vector and each prediction error are 
temporarily stored in the working area. The sum S m of the 
absolute values of the prediction errors is found before the 

15 process proceeds to step S214. 

[0145] In step S214, the sum S s of the absolute values of 
input quantized values, and S m and S s are compared with each 
other. If S s is less than S ra , the process determines that 
intra-encoding on the macroblock is efficient, and proceeds 

20 to step S204. As described above, processing for an intra- 

macroblock is performed. If not, the process proceeds to 
step S215 in order to perform inter-encoding. 
[0146] In step S215, a macroblock header is created for 
an inter-macroblock for motion compensation, and is stored 

25 in the MPEG-4 encoded data area. The process proceeds to 
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step S216. The created header includes encoded data of the 
motion vector calculated in step S213. In step S216, it is 
determined whether or not all the blocks of the macroblock 
have been processed. The process returns to step S202 when 
it is affirmatively determined, and proceeds to step S217 
when it is negatively determined. In step S217, the 
obtained prediction error is encoded for each block and is 
stored in the MPEG- 4 encoded data area. The process returns 
to step S202 via step S216 in order to perform processing of 
the next macroblock. 

[0147] As described above, the third embodiment is 
characterized in that a macroblock encoding mode is 
determined, and AC prediction is performed in the intra- 
frame encoding mode. 

[0148] According to the third embodiment, as described 
above, data encoded by JPEG encoding, which is an intra- 
frame encoding method, can be converted into data (I-VOP and 
P-VOP) encoded by MPEG- 4 encoding, which is an inter-frame 
encoding method, without reproducing pixel levels and 
deterioration. As a result, compared with JPEG encoding, it 
is expected that, in MPEG- 4 encoding, AC prediction greatly 
increases the encoding efficiency. Also, very high speed 
processing can be performed since tables can be used for 
code conversion. It is not necessary to use a memory for 
storing pixel levels. Thus, the configuration required for 
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processing can be reduced, thus enabling a reduction in 
apparatus cost. Moreover, the motion-vector retrieval 
requires a very small number of arithmetic operations, thus 
enabling a reduction in apparatus cost, and enabling high 
5 speed retrieval. 

[0149] In the second and third embodiments, forward 
prediction is only used as motion compensation. However, 
the present invention is not limited to the forward 
prediction, but another motion compensation mode in MPEG- 4 

10 encoding, for example, motion compensation based on 

bidirectional prediction, may be performed. 
[0150] In addition, in the first to third embodiments, 
cases in which JPEG encoded data were used as input and 
MPEG- 4 encoded data were used as output were described. 

15 However, the present invention is not limited to these cases, 

but encoding methods such as MPEG-1, MPEG- 2, H.261, and 
H.263 may be used. 

[0151] In other words, the foregoing description of 
embodiments has been given for illustrative purposes only 
20 and not to be construed as imposing any limitation in every 

respect. 

[0152] The scope of the invention is, therefore, to be 
determined solely by the following claims and not limited by 
the text of the specifications and alterations made within a 
25 scope equivalent to the scope of the claims fall within the 



- 40 - 



true spirit and scope of the invention. 



